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ABSTRACT 



This paper and the accompanying CD-ROM disks are the outcome 
of a federally funded project to research and develop an interactive 
multimedia sign language training system for educational interpreters for 
students with deafness. The program consists of four 20- to 30-minute lessons 
on the use of non-manual grammatical markers in American. Sign Lr.^gua g\. . 
Specific lessons are on question forms, topicalization, conditionals, and 
negation/assertion. The paper describes the procedures that were followed for 
designing, developing, and testing the interactive learning system. Features 
of the program include: (1) lessons provided on IBM-compatible CD-ROM disks; 

(2) textual instruction supplemented with video segments of fluent signers; 

(3) on-screen evaluation of the interpreter’s understanding of how non-manual 
markers are being used in the video segments; (4) immediate feedback to aid 
the learning process; (5) video intensive lessons that present multiple 
models of deaf signers; (6) an interactive tutorial that explains major 
features and how to navigate the program; and (7) student interaction with 
each video by pausing, stopping or replaying the video, and by playing the 
video at normal or slow-motion speeds. (CR) 
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BACKGROUND 



In July 1993, the U.S. Department of Education awarded a two-year 
grant to Subtle Impact Software, Inc. to research and develop an 
interactive multimedia sign language training system for 
educational interpreters for the deaf. The system was to 
incorporate sound, animation, graphics and video in an interactive 
computer environment to (1) strategically provide user 
participation and control; (2) fully exploit the powers of 
multimedia in providing sign language training; (3) isolate and 
intensify facial expressions, frozen forms, and body and gaze 
shifting; and (4) facilitate an evaluation-feedback capability. 
The final product was to consist of several lessons, with each 
lesson being approximately 30 minutes in duration. 

II. PROJECT OVERVIEW 



The project was divided into two phases. During Phase I (August 2, 
1993 - August 1, 1994), a prototype lesson was designed, developed 
and tested. During Phase II (August 2, 1994 - August 1, 1995), 
modifications were made to the prototype ' s concept and design based 
on the Phase I testing. A final lesson was then developed and 
tested and used as a template to develop additional lessons. 

The final product of this grant is a series of four lessons (titled 
Subtle Messages) on the use of non-manual grammatical markers in 
American Sign Language. Specific lessons are on Question Forms, 
Topicalization, Conditionals and Negation/Assertion. Although the 
lessons are primarily intended to support training of educational 
interpreters for the deaf, the lessons are beneficial to anyone who 
wants to learn or improve their American Sign Language skills. 
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III. PROJECT ACCOMPLISHMENTS 



The overall technical objective was to design and develop a 
computerized, interactive multimedia learning system to enhance the 
training of interpreters for the deaf. The following paragraphs 
describe the procedures that were followed for designing, 
developing and testing the interactive learning system and the 
specific accomplishments for each of the objectives and tasks 
outlined in the original and continuation grant applications. 

A. Phase I 



1 . Design a Multimedia Interactive Learning System 

(a) Task 1. Determine the learning objectives and the 
interactive learning experiences necessary in developing adverbial 
markers and clausal boundaries in sign language. 

A specific lesson was selected for prototype development. Subtle 
Impact wanted the lesson to be one that would be well suited for 
multimedia presentation and convincingly demonstrate the concept 
and feasibility. Therefore, the decision was made to develop the 
prototype lesson on "intonation in sign language" as expressed by 
the use of eyebrows, head tilt and body shift. The specific lesson 
objectives were: 

The interpreter will: 

( 1 ) Understand the concept of intonation as used in voice and 
sign language. 

( 2 ) Understand and be able to recognize how eyebrows help 
communicate intonation in questions. 
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(3) Understand and be able to recognize how head tilt helps 
communicate intonation for marking sentence boundaries and 
emphasizing words. 

(4) Understand and be able to recognize the use of body shift 
to mark clausal and sentence boundaries. 

(b) Task 2. Define interactive multimedia requirement 
specifications needed in satisfying the learning objectives, 
interactive learning experiences, and the necessary hardware 
constraints in making this an affordable, effective system. 

The initial requirement specifications were defined in terms of the 
interpreter training requirements, availability of computer 
equipment for playback, and the affordability of additional 
equipment needed to configure existing computer equipment. For the 
IBM and compatible configurations, the initial minimum requirements 
called for a multimedia-capable 386 computer with Windows 3.1 or 
higher, 100-200 megabytes available hard disk space, four megabytes 
of random access memory (RAM), and an SVGA monitor. However, as the 
program evolved, it became obvious that a more robust computer 
system would be required to effectively deliver interactive 
multimedia sign language training. The final minimum computer 
system capability required is a 486/66 Mhz with 8 MB of RAM, 
Windows 3.11 or higher and a SVGA monitor. The current series of 
four lessons require about 60 megabytes of hard disk space. With 
the rapid advances in technology and concomitant reduction in cost, 
this minimum system should be attainable by most agencies and 
individuals. 



(c) Task 3. Based on the requirement specifications , 
develop multimedia interactive design specifications. 

Based on the requirements for the program, a decision was made to 
use an authoring system that is specifically designed for 
interactive multimedia lesson development. After conducting an 
evaluation of existing authoring systems, we selected Authorware 
Professional as the most cost-effective system. The use of 
Authorware minimized the need for original programming but provided 
the capability to include the desired objects such as buttons, 
video, animation, sound, graphic displays. Since Authorware 
provided these capabilities, the specification of the design of the 
program became more closely aligned to the storyboard development 
process discussed under Task 5. 

(d) Task 4. Design computer-based training (CBT) 
modules, an interactive hypermedia (IH) interface and assessment 
feedback tools in accordance with the design specifications. 

The prototype was designed to provide the following features: 

( 1 ) The trainee had complete control of the pace of the 
lesson. The trainee was able to stop, pause, slow, reverse and 
restart each video segment. 

(2) Navigation through the lesson was facilitated by 
decreasing the number of options the trainee could select at each 
screen and by using strong visual cues and instructional text. 

(3) An evaluation/feedback component was included in the 
system for tracking an individual trainee's progress. 
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(e) Task 5. Storyboard and data flow chart the CBT 
modules, IH interface, and the necessary assessment tools needed in 
making the Subtle Impact multimedia interactive learning system an 
affordable, effective interactive multimedia system. 

Storyboards and flow charts were developed and approved by the 
Project Director. The flow charts were integral to the Authorware 
development process and provided a visual flow of activities for 
each segment and the entire program. Also, an evaluation/ feedback 
component was included in the system for tracking an individual 
trainee's progress. 

2 . Develop and Test Prototype 

Based on the initial storyboards, a pre-prototype multimedia lesson 
on intonation in ASL using the eyebrows, head tilt and body shift 
was developed. This lesson included most of the intended features 
of the prototype and enabled Subtle Impact to develop and evaluate 
the initial approach and a variety of multimedia techniques ( such 
as masking and full screen/partial screen, full-motion video) . 
This lesson was then evaluated by a consultant. Based on the 
consultant's evaluation and recommendations, the storyboards were 
revised, and technical changes were made to complete the prototype 
development . 

The prototype was then shown to faculty members of the Regional 
Interpreter Training Program, Front Range Community College. In 
general, they did not agree with the approach that was used to show 
how eyebrows, head tilt and body movement are used to provide 
intonation (each feature was presented separately) . Since these 
features are used together for intonation, they suggested we shift 
our focus away from "intonation" to a more traditional approach 
which focuses on how these non-manual markers are used to indicate 
sentence types (wh-, yes /no, etc.). The program was then revised 
based on this recommendation and then demonstrated to Front Range 
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faculty members again. They agreed that the program has excellent 
potential for interpreters who have not attended formal interpreter 
training programs and could be used to complement existing 
interpreter training programs. 

The program was then evaluated by educational interpreters 
attending a workshop at the Colorado School for the Deaf and the 
Blind and sign-language trainees and staff members of the Pikes 
Peak Center on Deafness. Additionally, the program was 
demonstrated at the "Educational Application of Technology for 
Persons with Sensory Disabilities Symposium, " which was sponsored 
by the U.S. Department of Education/OSEP and the National Technical 
Institute of the Deaf (NTID) ? Rochester, NY; July 20-22, 1994. 

The evaluation results reinforced the belief that interactive 
multimedia learning systems have great potential for training 
educational interpreters and also indicated that multimedia systems 
have additional benefits for teaching sign language to anyone. The 
evaluation also resulted in several recommendations for improving 
the program. 

B. Phase II 

Following an analysis of the Phase I evaluation. Subtle Impact 
reviewed and revised the system concept as necessary. The system 
was redesigned based on the lessons learned and Phase II 
development was initiated. During Phase II, Subtle Impact 
developed a complete multimedia interactive learning system and 
performed extensive tests of the final product. The goal was to 
have four market-quality lessons at the end of Phase II. 



1 . Redesign the Multimedia Interactive Learning System 



(a) Task 1 & 2. Redesign the prototype, and develop the 
full-scale version of the lesson. 

Based on the results of the Phase I evaluation. Subtle Impact 
redesigned the prototype. The redesign required a repeat of the 
design steps that were accomplished in Phase I. Specifically, this 
task required review and as-required modification of the lesson 
objectives, requirement specifications, design specifications, CBT 
modules and the storyboards. The redesigned prototype then became 
the first production lesson (Question Forms) and served as the 
model (template) for additional lessons. 

The full-scale version was developed using the Authorware 
Professional authoring system based on the revised storyboards. 
This version was of production quality and incorporated text, full- 
motion video, graphics, animation and sound. Although the product 
was developed primarily for hearing interpreters, it was 
intentionally developed to be usable without sound for the benefit 
of the deaf and hard-of-hearing. After in-house testing and 
acceptance, the program was produced in limited numbers on CD-ROM 
disks to support beta testing. 

To enhance our performance in Phase II, a closer working 
relationship was established with the Pikes Peak Center on Deafness 
(PPCOD) . Under this relationship, the PPCOD defined the lesson 
objectives, content and storyboards; provided the deaf ASL models; 
and participated in the videotaping and the testing for each 
lesson. With this significant help in program content development 
from the PPCOD, Subtle Impact was able to concentrate more on 
resolving the technical aspects of the program (structure) and 
integrating the content into this structure. This enabled Subtle 
Impact to simultaneously work on the completion of four lessons. 
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These four lessons were defined to be Question Forms, Conditionals, 
Topicalization, and Assertion/Negation. 

(b) Task 3. Conduct Beta Test/Field Evaluation. 

Copies of the program were distributed on CD-ROM disks to both 
hearing and deaf persons for beta testing and field evaluations . 
The primary purpose of the beta tests was to evaluate the program's 
operability, functionality, and content. Test guidelines and 
questionnaire/report forms were developed to support the tests. 

The original grant application proposed the use of the Educational 
Interpreter Performance Assessment (EIPA) process with interpreter 
trainees to evaluate the effectiveness of the iterative multimedia 
learning system. However, by the end of the grant, significant 
"bugs" continued to be found in the program, and the program was 
never stable enough to use the EIPA process. Therefore, all tests 
and evaluations were accomplished by the Colorado School for the 
Deaf and the Blind; The Pikes Peak Center on Deafness; and Sign 
Language Consultants, Inc., McLean, VA. 

(c) Task 4. Fix and Retest. 

Problems discovered during the tests were corrected, and the 
program was retested to insure the problems were actually resolved. 
However, many of the fixes were made after the completion of the 
grant . 



(d) Task 6. Convert the program to run on the Macintosh 

computer. 

Because of the "bugs" encountered in the IBM PC version. Subtle 
Impact was not able to convert the program to run on the Macintosh 



computer during the period of the grant. However, this conversion 
is planned for future work. 

C. The Final Product - Subtle Messages 

The final product of this grant is a series of four lessons on the 
use of non-manual grammatical markers in American Sign Language. 
The series is titled Subtle Messages because the non-manual 
markers used by fluent deaf signers are often quite subtle. 
Specific lessons are on Question Forms, Topicalization, 
Conditionals and Negation/Assertion. Although the lessons are 
primarily intended to support training of educational interpreters, 
the lessons are beneficial to anyone who wants to learn or improve 
their American Sign Language skills. 

Specific features of Subtle Messages are as follows: 

1 . The user is provided lessons on Compact Disk-Read Only 
Memory (CD-ROM) disks. 

2. Lesson objectives are satisfied through the use of textual 
instruction supplemented with video segments of fluent signers, on- 
screen evaluation of the interpreter's understanding of how non- 
manual markers are being used in the video segments, and the use of 
immediate feedback to aid the learning process. 

3. All lessons are video intensive and present multiple 
models of deaf signers. Subtle Messages contains over 700 video 
clips; therefore, a student can complete each lesson five times 
without seeing the same video twice. Eight different models from 
the local community are used in the program to give the student 
exposure to differences in signing among deaf signers. 
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4. The lessons are intuitive and easy to use. However, an 
interactive tutorial is included in Subtle Messages to explain the 
major features of the program and how to navigate through it. 

5. The student can interact with each video by pausing, 
stopping or replaying the video and playing the video at normal or 
slow-motion speeds. This interaction enables the student to focus 
on the specific non-manual markers. 

6. Each lesson is normally 20-30 minutes long; however, the 
lessons are student controlled, allowing the student to learn at 
his/her own pace. 

D. Dissemination Activities. 

Following are the activities used to disseminate information 
concerning the grant and its resultant product. The activities 
include those used during and after the period of the grant, as of 
the date of this report. 

1. Subtle Impact representatives presented/exhibited at the 
following meetings and conferences during the period of the grant: 

(a) The Boys Town National Research Hospital 8th Annual 
Conference on Issues in Language and Deafness; Boys Town National 
Research Hospital; Omaha, NE (October 1993) 

(b) The National Technical Institute of the Deaf (NTID) 
National Symposium on Educational Applications of Technology for 
Persons with Sensory Disabilities; Rochester, NY (July 1994) 

(c) The Conference of Interpreter Trainers Tenth 
National Convention; Charlotte, NC (October 1994) 
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(d) The Boys Town National Research Hospital 9th Annual 
Conference on Issues in Language and Deafness; Nebraska City, NE 
(October 1994) 

(e) Colorado's 1994 Symposium on Deafness: Colorado 
Springs, CO (November 1994) 

(f) The California State University, Northridge (CSUN) 
Tenth Annual Conference: "Technology and Persons with Disabilities" 
(March 1995) 

(g) The Project Director's Conference in Washington, 
D.C . (July 1995) 

(h) The 14th Biennial National Convention of the 
Registry of Interpreters for the Deaf; New Orleans, LA (August 
1995) 



(i) The "Technology for Kansans Project"; Topeka, KS 
(September 1995) 

(j) Colorado's 1995 Symposium on Deafness: Copper 

Mountain Resort, CO (October 1995) 

2. Mailings were sent to over 2,000 agencies and individuals 
who may have an interest in Subtle Messages . Each mailing included 
a letter and brochure which explained the background, features and 
benefits of the program. A post card, to be used to order a Subtle 
Messages demonstration CD-ROM was also included. The recipients of 
the mailings included schools and local programs, university and 
college programs (interpreter and teacher training programs), 
professional service organizations and state and local government 
agencies . 
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IV. SUMMARY 



A two-year grant was awarded to Subtle Impact Software, Inc. in 
July 1993 to research and develop a multimedia interactive learning 
system to enhance the training of educational interpreters for the 
deaf. This grant has resulted in the development of four 
interactive multimedia lessons on the use of non-manual grammatical 
markers in American Sign Language. These lessons are on Question 
Forms, Topicalization, Conditionals and Assertion/Negation; are on 
CD-ROM disks; and are available for purchase as a package. The 
current lessons are usable on IBM and compatible computers; 
however, a Macintosh version will be available in the future. 
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